Author response to: “Decoding Face Exemplars from fMRI Responses: What Works, What Doesn’t?” by J. Carlin (2015) fMRI MVPA can decode some variables better than others, and clustering is a likely culprit

In our recent study in The Journal of Neuroscience (Dubois, de Berker, & Tsao, 2015), we compared the amount of information about face identity and face viewpoint that can be retrieved from populations of single units and from functional magnetic resonance imaging (fMRI) voxel patterns with a simple linear classifier in the macaque monkey. Based on our decoding results and on further probing of properties of the single-unit representations, we concluded that multi-voxel pattern analysis (MVPA) of fMRI data was not able to extract information from underlying single-unit populations that were poorly clustered with respect to their selectivity within the dimension of interest. Carlin (2015) challenges our conclusion, arguing instead that poor functional signal-to-noise ratio (fSNR) can account for our results. We welcome the opportunity to clarify the arguments that led us to discard the fSNR account, and address Carlin (2015)'s arguments point-by-point below. Carlin (2015) first points out that identity can be decoded above chance in ML/MF despite the low identity clustering for neurons in this region. While this may appear to weigh against the clustering explanation and in favor of an fSNR account (as Carlin (2015) rightly notes, ML/MF has the highest fSNR in the fMRI data), the complication here is that fMRI is not actually picking up identity information, but a low-level confound. The output of the classifier for ML/MF (Fig. 6) is very similar to the output for V1 (Fig. 8): above chance performance is driven by ID5, who was bald and generally brighter than the other identities. What may be confusing is that this confound did not artificially inflate identity clustering in ML/MF (Fig. 7): this is because clustering was established on the basis of all 25 identities in the face views image set (Freiwald & Tsao, 2010), hence drowning the influence of ID5 and more closely reflecting identity clustering than low-level confounds. Carlin (2015)'s second argument is that, if clustering is the sole determinant of successful fMRI decoding, then we should get a similarly successful readout of the underlying neural population information for viewpoint and identity in a patch that has similar clustering for these two dimensions. Carlin (2015) points to AM as having this property, and then argues that since identity is not read out well whereas viewpoint is, clustering cannot be the single explanation. But viewpoint and identity clustering in AM are not identical: by our proxy measure, viewpoint clustering appears ~50% …